Field
The invention relates to a method for improving operation of at least one robot which is operated on the basis of a set of predefined actions.
Description of the Related Art
Over the last years, robots have been improved so that the tasks that can be delegated to a robot are not limited to a simple task like picking and placing items from a predetermined position to a predetermined target position. The robots now often use situation recognition on the basis of which it is decided which action is to be taken next. But still the overall behavior of the robot typically is composed of distinct actions. As an example, motions can be structured as smaller motion chunks, sometimes represented as so-called motion primitives. It is to be noted that in the context of the present invention and also the discussion of prior art, the term “action” is not limited to motion. The term “action” rather defines any actuation of a robots component, such as for example the robots joints, manipulators or sensors for certain amount of time or until a certain event.
The actions that are used by a robot in order to perform a particular task or a desired behavior usually are created by a human designer or learned individually. Such single actions are then executed on the robot in a sequence or as a reaction to the perceived state. Although, the task that is delegated to the robot may be achieved by such a sequence of actions it is often desirable to let the robot perform multiple actions in parallel to achieve different goals at the same time. One example may be that individual motion primitives may affect separate end-effectors of a robot, for example the arms of a humanoid robot. It is evident that by performing multiple actions in parallel, the efficiency could be improved by reducing the time needed to achieve the combined goal. FIGS. 5 and 6 show such a situation. The scenario shows a humanoid service robot 10 and the task is to clean up a table by picking up an empty cup 14 and placing it onto a tray 13 for bringing them away later. Since the position of the tray 13 may vary, it might be necessary to push the tray 13 aside, e.g. to the center of the working range of the robot 10, so that placing the cup 14 on the tray 13 becomes possible without violating joint limits. A well-known strategy would now be to conduct different actions sequentially, which means that first the tray 13 is pushed by the first end-effector 11 towards the center of the working range of the robot 10 and thereafter the cup 14 is picked and placed by the other end-effector 12 of the robot. This can be performed now because the tray 13 is positioned in an area where the robot 10 can place the cup 14.
As it is now shown in FIG. 6, it would be desirable to execute a combined action of pushing the tray 13 to the center and picking the cup 14 with the other hand (end-effector).
In order to increase the efficiency of a working robot, it was described by A. Gil, H. Stern, Y. Edan and U. Kartoun in “A Scheduling Reinforcement Learning Algorithm” (International Symposium on Flexible Automation, Atlanta, Ga., 2008) that using reinforcement learning on the basis of Q-learning is used to calculate rewards that lead to a preference of faster task completion. Thus, the control policy which is the outcome of such reinforcement learning process, has an improved efficiency with respect to an overall time for completing a task. But nevertheless, it makes only use of sequentially executed actions.
On the other side, it is described in U.S. Pat. No. 8,265,789 B2 how two actions can be executed at the same time. But the reference is silent with respect to the generation of such a combined action because it is limited to the question of how two actions are executed at the same time, e.g. blending of motion commands. Thus, if the combined action is to be executed the reference gives the teaching of how this can be implemented in the robot. Last but not least, it is described by F. Zacharias, D. Leidner, F. Schmidt, C. Borst and G. Hirzinger in “Exploiting Structure in Two-armed Manipulation Tasks for Humanoid Robots” (IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Taipei, 2010) that a two-arm-robot can parallelize two actions based on the required workspace. In order to make the two actions being executed in parallel, it is determined if the workspace for these two actions is disjunct. In that case the tasks are independent and two individual motion planners are then used to generate the appropriate motions individually and the actions are finally executed in parallel. This approach only takes into consideration geometric aspects of the two actions. It furthermore does not assess the question if a combination of two actions is feasible at all.
It is thus the object of the present invention to provide a method that improves the operation of at least one robot which is operated on the basis of predefined actions.